
DOSSIER 

l|RNATIONAL A FFLICTION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 

(19) World Intellectual Property 
Organization 

International Bureau 




(43) International Publication Date 

10 September 2004 (10.09.2004) PCT 



(10) International Publication Number 

WO 2004/077436 Al 



(51) International Patent Classification 7 : 

H04N 7/24 

(21) International Application Number: 



G11B 27/10, 



PCT/IB2004/050135 

(22) International Filing Date: 20 February 2004 (20.02.2004) 
(25) Filing Language: English 



(26) Publication Language: 



English 



(30) Priority Data: 

03075566.4 
03103263.4 



25 February 2003 (25.02.2003) EP 
1 September 2003 (0 1 .09.2003) EP 



(71) Applicant (for all designated States except US): KONIN- 
KLUKE PHILIPS ELECTRONICS N.V. [NL/NL]; 
Groenewoudseweg 1, NL-5621 BA Eindhoven (NL). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): NEWTON, Philip, 

S. [NL/NL]; c/o Prof. Holstlaan 6, NL-5656 AA Eind- 
hoven (NL). KELLY, Declan, P. [IE/NL]; c/o Prof. Hol- 
stlaan 6, NL-5656 AA Eindhoven (NL). VAN GESTEL, 
Wilhelmus, J. [NL/NL]; c/o Prof. Holstlaan 6, NL-5656 
A A Eindhoven (NL). 



(74) Agent: DE HOOG, Johannes, H.; Prof. 
NL-5656 AA Eindhoven (NL). 



Holstlaan 6, 



(81) Designated States (unless otherwise indicated, for every 
kind of national protection available): AE, AG, AL, AM, 
AT, AU, AZ, BA, BB, BG, BR, BW, BY, BZ, CA, CH, CN, 
CO, CR, CU, CZ, DE, DK, DM, DZ, EC, EE, EG, ES, FI, 
GB, GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, KE, 
KG, KP, KR, KZ, LC, LK, LR, LS, LT, LU, LV, MA, MD, 
MG, MK, MN, MW, MX, MZ, NA, NI, NO, NZ, OM, PG, 
PH, PL, PT, RO, RU, SC, SD, SE, SG, SK, SL, SY, TJ, TM, 
TN, TR, TT, TZ, UA, UG, US, UZ, VC, VN, YU, ZA, ZM, 
ZW. 

(84) Designated States (unless otherwise indicated, for every 
kind of regional protection available): ARIPO (B W, GH, 
GM, KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZM, ZW), 
Eurasian (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), Euro- 
pean (AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, FI, FR, 
GB, GR, HU, IE, IT, LU, MC, NL, PT, RO, SE, SI, SK, 
TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, 
ML, MR, NE, SN, TO, TG). 

Declaration under Rule 4.17: 

— as to applicant's entitlement to apply for and be granted 
a patent (Rule 4.17(H)) for the following designations AE, 
AG, AL, AM, AT, AU, AZ BA, BB. BG, BR, BW. BY, BZ, 
CA, CH, CN, CO, CR. CU, CZ, DE. DK. DM, DZ, EC, EE, 
EG, ES, FI, GB, GD. GE. GH, GM. HR. HU, ID. IL, IN. IS, 
JP, KE, KG. KP. KR, KZ, LC, LK, LR, LS, LT. LU, LV, MA, 
MD. MG. MK. MN. MW, MX. MZ. NA, NI, NO, NZ, OM, 

[ Continued on next page] 



(54) Title: DATA MEDIUM 



4 



6 



(57) Abstract: Data medium adapted to triggering applications in response to an event. The data medium comprises a plurality of 
play -items and at least one event-item containing an even function call. Further the data medium comprises a play-list for determining 
the order, in which the play-items and the event-item are read out of the data medium. 



BNSDOCID: <WO_ 



„2004077436A1_L 



WO 2004/077436 Al I lllll llllilll II DIIII Hill Hill lllll llll I II III lllll IIKI Hill lllll Mill III! Illllll illl IIII IIII . 



PG. PH, PL, PT, RO. RU. SC, SD. SE. SG, SK, SL. SY, 77. 
TM. 77V, 77?, TT, rz. UA, UG, UZ, VC. VN, YU, ZA, ZM. 
ZW, ARIPO patent (BW, GH, GM, KE, LS. MW, MZ. SD. 
SL, SZ, TZ. UG, ZM, ZW), Eurasian patent (AM, AZ, BY. 
KG, KZ, MD. RU, TJ, TM), European patent (AT, SE, BG. 
C/7. CY, CZ, DE, OK, EE. ES, El, ER, GB, GR, HU, IE. IT. 
LU, MC. NL. PT, RO, SE. SL SK. TR), GAP I patent (BE, BJ. 
CE CG. CI, CM, GA, GN, GQ, GW, ML, MP, NE, SN, TD, 
TG) ' 



Published: 

— with international search report 

For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



BNSDOCID: <WO 2004077436A1_I_> 



WO 2004/077436 



PCT/IB2004/050135 



Data Medium 



The present invention is concerned with data media adapted to triggering 
applications in response to an event, methods for triggering applications in response to an 
event and devices adapted to reading and processing the data from a data media adapted to 
triggering applications . 

5 

The development of data carrier for providing video and/or audio content to a 
consumer has been marked by an ever increasing demand in the storage capacity of such data 
carriers and the demand for interactivity. In order to meet these demands new data media are 
being developed in particular blue-ray discs. Furthermore more and more interactive 
10 applications are being introduced to such storage media. A new publishing format is being 
designed for blue-ray discs which allows picture quality in HDTV resolution and improved 
interactivity. 

A particular desire is to provide a synchronization between video content and 
a computer application such as a Java application, such that the application is triggered, if an 
15 event occurs in the content of a video during playback. For example, if a movie reaches an 
important scene during playback, an application is triggered which displays additional 
information on the scene from the director. Another example for an event can be the scoring 
of a goal, if a sports match is being displayed. The application can be triggered to change the 
score of the match. 

20 

The prior art document WO 02/05104 addresses this above mentioned 
problem but fails to disclose a workable solution. WO discloses on page 61 the use of events, 
which are integral to synchronizing DVD-Video with other media. But the document does 
25 not disclose how different events in particular time events in the content of the DVD may 
trigger an application. 
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It is therefore object of the present invention to provide a solution for 
triggering an application in response to the play-back of an event in the content of a data 
medium. 

The problem is solved by a data medium adapted to triggering applications in 
5 response to an event comprising a plurality of play-items, at least one event-item containing 
an event function call and a play-list for determining the order, in which the play- items and 
the event-item are read out of the data medium. The play-items correspond to the applications 
and data contained on the data medium, among which a user of the data medium can choose. 

The triggering of the application can be effected by the user of the data 
10 medium, who may choose from the play-list which play- items shall be played back from the 
data medium. Further triggering may be effected by changing the state of the data medium. 
An example for the change in the state of the data medium is a change of a playback rate of 
the data medium. The play list may be programmed in this case to immediately execute the 
event-item containing the event function, which in turn triggers the application. 
15 If there is no special request from the user of the data medium, the play-list 

will read out the play- items in a predetermined order. The event- item will be read at the 
beginning, end or in between any two successive play- items. The application is triggered in 
response. Therefore the event triggering the application is the beginning of playback, the end 
of playback or the change between two play-items. The event triggering the application may 
20 be defined as a change in the play- items being played back form the disc. The event-item 

contains event information relating to the change between the play-item preceding the even- 
item and/or the play- item succeeding the event-item. 

Different event function calls may correspond to different events whereupon 
the event-item is read from the data medium. Therefore, the nature of the event may be 
25 discerned by the kind of function which is being called. Different events may call for 
different functions. For example the event of a goal in a game calls for a function for 
determining the score of a game. The event of a scene change in a movie may call for an 
event function for displaying additional information about the movie in particular the scene 
displayed. The application to be executed must therefore be known at the time of the 
30 authoring of the data medium, if the data medium may only be written once. 

According to another embodiment the even-item contains event information 
relating to the event whereupon the event- item is read out of the data medium. The function 
being called in response to different events may be the same. The information contained in 
the event-item may be transferred to the function. This event information may in turn be used 
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to trigger specific applications. The nature of the event may be easily discerned by the calling 
application, which in turn may execute further applications in response to the event 
information received. 

The data medium is preferably a blue-ray disc, since blue-rays provide for 

5 play-lists and play items according to the international standard ISO/IEC 1381 for blue-ray 
disc publishing. The event function call may be adapted to registering to a computer 
application, in particular a Java application, an HTML application, an XML application or a 
C-application. The play-items typically contain audio and/or video data. 

The problem of the invention is solved in particular by a method for triggering 

0 applications in response to an event characterized by the steps reading event-items and play- 
items from a data carrier in the order determined by a play-list stored on the data carrier and 
executing an event function in response to reading an event function call contained in the 
event item. This is the method, which must be carried out by a storage medium reading 
device, which is adapted to reading the data medium previously discussed. 

5 Another solution to the problem of the invention is a data medium comprising 

at least one clip-stream file having a plurality of first elementary streams and at least a second 
elementary stream, wherein the first elementary streams are video and/or audio streams and 
the second elementary stream is a private stream containing code segments. The coding 
mechanism must allow private streams to be embedded in the data stream, which is to be 

0 decoded. Such a coding mechanism is provided for example by the MPEG2 standard for 
coding video and/or audio-data. The first elementary streams form a contiguous data stream, 
if read from the data medium. MPEG2 multiplexes the first elementary streams, which 
contain audio and/or video data. The private stream contains data which are not encoded 
according to the MPEG2 standard. The code segments may contain Java class file, which 

5 when loaded run in the Java virtual machine and can be used to display information to the 
user on the event. Furthermore the code segments may constitute function calls which 
transfer data about the event and the time of the event to the application registered to the 
function call. 

Preferably the first elementary streams are encoded in such a way that the first 
0 elementary streams are multiplexed to form a coherent clip-stream, when read from the data 
medium. The normal play time is linked to the clip-stream and the normal play time is 
indicative of the time in the clip-stream. Normal play time is a time line linked to the 
coherent clip-stream. The timeline is continuous over the duration of the clip stream. The 
normal play time refers to the real time of the stream content regardless of the mode of it's 
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presentation. For example, when a video is played in reverse normal play time counts down 
rather than up and when a video is played at ten times the speed normal Play time progresses 
at ten times the regular rate. Therefore a point in time in normal playtime is always indicative 
of a certain, unalterable event in the content of the clip stream. An application can be 
5 triggered in response to an event in the content, if the point in time in normal play time is 
known, at which the event takes place. Therefore the code segments may contain simply the 
time in normal play time and the name of the event taking place. This information can be 
transmitted before the event actually takes place. Triggering the application in response to an 
event may simply be achieved by monitoring the normal play time of the clip-stream. 

10 The point in time in normal play time may be determined by any data medium 

drive which has a system clock. The length of the clip-stream in normal play-time 
corresponds to the number of clock cycles needed to display the clip stream at a 
predetermined "normal" speed. If the clip is displayed faster, the normal playtime may be 
calculated from the difference in speed. 

15 A stream event descriptor is a function call which contains the data relating to 

an event in the content of the stream and a point in time of the event in the normal play-time. 
By implementing stream event descriptors on the data medium, the triggering of the 
application may be achieved synchronously with the occurrence of an event in the content of 
the data medium. 

20 The second elementary streams may be encoded in such a way that the second 

elementary streams are multiplexed to form a coherent private-stream, when read from the 
data medium. The normal play-time is linked to the private stream and the normal play-time 
is indicative of the time in the private-stream. If the private stream and the clip stream are 
processed simultaneously, the code segments in the private stream having the same point in 

25 time in normal play-time will be executed simultaneously with the events in the clip-stream. 
Thus applications, which are registered to the code segments, are executed simultaneously 
with events in the clip stream. The code segments preferably constitute function calls for 
example in Java. 

The solutions of embedding a function call in a play list and in a clip stream 
30 file may be combined with each other. Depending on the kind of event, the synchronization 
of the event can be achieved by including a function call in the clip stream file or in the play 
list. If an application is only related to a single piece of content, it is typically event driven 
and embedding a function call in the clip stream file is the preferred solution. Applications 
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that deal with multiple clip streams such as an application that allows a user to view multiple 
angles of a camera shot are typically linked to the play-list. 

The applications linked to events in the content of a data medium according to 
the invention may also access data on the data medium such as image frames. The data to be 
5 accessed may also he embedded in clip stream files as private streams. The private stream 
may represent images or even HTML or XML files. The normal transfer of data between the 
CPU and a peripheral device may be used. A set of APIs (Application Protocoll interface) for 
file read and write access may be implemented, in order to allow the application to read data 
from the disc, APIs are implemented by writing function calls in the program, which provide 
10 the linkage to the required subroutine for execution. 

These and other aspects of the invention will be apparent from and elucidated 
with reference to the embodiments described hereinafter. The preferred embodiments of the 
present invention are discussed by way of example with reference to the accompanied 
drawings. 

15 



Fig. 1 shows the data structure of a data medium according to a first 
embodiment of the present invention, 

Fig. 2 shows the first and second elementary streams of a clip-file contained in 
20 the data medium according to a second embodiment of the present invention before 
multiplexing, 

Fig. 3 shows the private stream and the clip-stream after multiplexing 
according to the second embodiment of the present invention. 



25 

The data structure shown in Fig. 1 consists of a play-list 1, which is associated 
with a number of play-items 2 and an event item 3. The play-items 2 are each associated with 
a respective clip info file 4. The clip info files contain information about respective clip- 
stream files 6. Information about the coding conditions of the stream files and a mapping of 
30 the normal play time and the location in the stream may also be included in the clip info file. 
They indicate in particular where the clip stream data is stored on the data medium. 
Audio/Video data is stored in the clip-stream files 6 according to the MPEG-2 transport 
stream format used in digital TV broadcasts. The transport stream format provides for a 
better error correction than the standard program stream format. Associated with each clip 
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stream file 6 is a clip info file , which stores additional information about the audio/video 
data to support playing back of the clip. Separate from the clip layer is the play list layer, 
which defines the actual play list the user can play. Each play list 1 comprises one or more 
play items 2 and each play item points to a continuous piece of MPEG-2 transport stream in a 
5 clip file. 

The play file also points to an event item 3. The play list determines the order, 
in which the items 2 and 3 are read out of the data medium. The event item 3 may be 
executed in any chosen order according to the play list 1 . The event item is connected to an 
event info file 5, which contains information about the location and kind of data stored in the 

10 event file 7. The event file 7 contains a specific function call to be executed. At the time the 
event item 3 is activated by the play list 1, the function call is executed. As a consequence an 
application registered to the function call is executed. 

In Fig. 2 a number of the first elementary streams 20 is depicted. The first 
elementary streams 20 contain audio/video data encoded according to the MPEG-2 standard. 

15 Furthermore, a second elementary stream 21 is depicted in Fig. 2. The first and second 
elementary streams are associated with a clip info file 4 (not shown). The clip info file 
contains information about the location of the elementary streams 20 and 21 on the disc and 
information about the nature of the elementary streams (MPEG-2 or private stream). The 
second elementary stream 21 is a private stream which contains a function call and the time 

20 of the function call in normal play time (NPT). 

In Fig. 3 the first and second elementary streams 20 and 21 are shown after 
multiplexing according the MPEG2-standard. The first elementary streams now form a 
contiguous clip stream file of video and or audio data. The normal play time (NPT) 
associated with the clip stream is depicted on the x-axis below the clip stream. The only 

25 second elementary stream forms the private stream shown above the clip stream. The position 
of the first and second elementary stream with respect to the x-axis determines the time when 
the respective streams are played back (clip stream) or executed (private stream). Therefore 
the function call contained in the private stream is executed simultaneously with an event 
contained in the region of the contiguous clip stream below the second elementary stream. 

30 It is apparent to the person skilled in the art, that the previous embodiments of 

the invention are only exemplary and that the invention may be carried out in ways differing 
from the previous embodiments. The claimed invention is defined by the appended claims. 
Reference signs in the claims shall not be construed as limiting the scope of protection. 
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Data medium adapted to triggering applications in response to an event 
a plurality of play-items (2), 

at least one event-item (3) containing an event function call and 
a play-list (1) for determining the order in which the play-items (2) and the 
event-item (3) are read from the data medium. 

2. Data medium according to claim 1, characterized in that the different event 
function calls correspond to different events whereupon the event-item (3) is read from the 

10 data medium. 

3. Data medium according to claim 1 or 2, wherein the even-item (3) contains 
event information relating to the event whereupon the event- item (3) is read from the data 
medium. 

15 

4. Data medium according to claim 1, characterized in that the event-item (3) 
contains event information relating to the change between the play-item (2) preceding the 
even- item (3) and/or the play- item (2) succeeding the event-item (3). 

20 5. Data medium according to one of the previous claims, characterized in that the 

play-list (1) is adapted to changing the predetermined order, in which the play-items (2) and 
the event-item (3) are to be read from the data medium, if a state of the data medium changes, 
wherein the event-item (3) is to be read first and the event-item (3) contains event 
information relating to the change of the state of the data medium. 

25 

6. Method for triggering applications in response to an event characterized by the 

steps, 

reading event-items (3) and play- items (2) from a data carrier in the order 
determined by a play- list (2) stored on the data carrier, 



1. 

comprising 

5 
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executing an event function in response to reading an event function call 
contained in the event item (3). 

7. Data medium comprising at least one clip-stream file having a plurality of first 
elementary streams (20) and at least a second elementary stream (21), wherein the first 
elementary streams (20) are video and/or audio streams and the second elementary stream 
(21) is a private stream containing code segments. 

8. Data medium according to claim 7, characterized in that the first elementary 
streams (20) are encoded in such a way that the first elementary streams (20) are multiplexed 
to form a coherent clip-stream, when read from the data medium, a normal play time (NPT) 
is linked to the clip-stream and the normal play time (NPT) is indicative of the time in the 
clip-stream. 

9. Data medium according to claim 8, characterized in that the second elementary 
stream (21) contains a stream event descriptor having data relating to an event in the content 
of the stream and a point in time of the event in the normal play-time (NPT). 

10. Data medium according to claim 8, characterized in that the second elementary 
streams (21) are encoded in such a way that the second elementary streams (21) are 
multiplexed to form a coherent private-stream, when read from the data medium, the normal 
play-time (NPT) is linked to the private stream and the normal play-time (NPT) is indicative 
of the time in the private-stream. 

1 1 . Device adapted to reading and processing the data from a data medium 
according to one of the claims 7 to 10. 
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